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Benchmarking:  A  surveyor's  mark  (...)  of  previously  determined  position  (...)  and  used  as 
reference  point  (...)  standard  by  which  something  can  be  measured  or  judged. 

-  Webster's  Dictionary 


1.  Innovation  Benchmarking 


The  pioneering  work  of  Keams  at  Xerox  Corporation  on  benchmarking  in  management  used  the 
following  working  definition: 

"Benchmarking  is  the  continuous  process  of  measuring  products,  services  and 
practices  against  the  toughest  competitors  or  those  companies  recognized  as 
industry  leaders."  ' 

Later,  a  broader  understanding  in  terms  of  action-oriented  concepts  led  to  the  rather  different 
approach  of  Camp:  "Benchmarking  is  the  search  for  industry  best  practices  that  lead  to  superior 
performance."^ 

Although  benchmarking  is  relatively  new,  it  is  quite  well  established.  The  main  problem  with 
benchmarking  is  that  most  people  use  rather  crude  scores  to  carry  out  benchmarking  comparisons. 
The  purpose  of  this  paper  is  to  return  to  the  original  meaning  and  to  propose  a  different  type  of 
benchmarking  using  quantitative  measures,  in  order  to  position  a  product  or  service  in  terms  of  its 
technology. 

This  paper  concentrates  on  the  benchmarking  of  innovations,  not  of  standard  products  or 
practices.  This  is  not  a  critical  remark  toward  standard  benchmarking  —  just  another  approach  to 
achieving  the  same  goal.  In  general,  whenever  one  can  use  quantitative  data  organized,  for 
example,  as  a  table  of  data,  one  is  better  off  than  when  qualitative  judgments  —  the  conventional 
approach  in  benchmarking  —  are  made. 

The  structure  of  this  paper  is  as  follows:  The  first  section  explains  how  technological 
benchmarking  can  be  carried  out  for  strategic  positioning  of  firms  in  global  telecom  markets 
(telecom  manufacturers).  Second,  overall  positioning  of  firms  in  the  information  technology 
market  is  discussed.  Third,  the  way  in  which  knowledge  production  leads  to  innovation  and 
growth  is  explored.  Fourth,  specific  positioning  of  firms  in  single-product  quality  within  the  area 
of  telecom  products  is  examined.  Finally,  the  last  section  provides  a  typology  of  firms  based  on 
how  well  product  quality,  measured  by  the  "technometric"  approach,  correlates  with  market-based 
preferences. 

The  methodological  tools  used  for  this  quantitative  benchmarking  begin  with  patent  statistics.  But 
benchmarking  is  often  not  unidimensional,  but  rather  multidimensional.  So  we  need  to  use  new 
tools  to  express  fully  the  multidimensional  nature  of  quality  (strategic  markets,  strategic 
sub-technologies),  such  as  multidimensional  scaling  (section  2). 

Patent  statistics  are  also  useful  for  exploring  the  knowledge  production  that  leads  to  innovation 
and  subsequent  growth  (section  3).  A  technique  known  as  technometric  benchmarking  is  applied 


'  D.T.  Kearns.  Quality  Improvement  Begins  at  the  Top,  in  Bowies,  ed..  World  20  (5),  pp.  21,  1986.  The  concept 
dates  back  to  around  1979. 

■  R.C.  Camp.  Benchmarking:  the  search  for  industry  best  practices  that  lead  to  superior  perfommnce.  Milwaukee: 
ASQC  Quality  Press,  1989,  p.  12. 


to  give  quantitative  expression  to  the  multidimensional  nature  of  most  products  and  services,  i.e.  to 
product  quality  (section  4). 

For  most  people,  a  patent  is  a  legal  document.  But  what  interests  us  in  patent  statistics  is  the 
quality  of  the  output  of  knowledge  that  finds  expression  in  patents.  If  in  a  company  two  engineers 
work  for  a  year  on  a  defined  project  funded  from  internal  sources  (cash  flow),  and  if  they  are 
successful  and  invent  something  new,  then  we  eventually  have  a  document  emerging  from  this  lab 
that  tells  us  that  two  engineers  worked  for  a  while  on  a  certain  invention,  described  very  precisely. 
We  can  read  from  the  document,  as  we  read  from  scientific  publicafions,  that  this  company  has 
deliberately  brought  about  a  certain  inventive  step,  now  documented  and  codified.  So  patent 
documents  point  to  those  areas  of  activity  in  which  a  company  has  invested  R&D  labor  and 
resources.  When  patent  examiners  (in  most  countries  civil  servants  at  patent  offices)  discover  that 
the  idea  is  not  new  -  but  is  already  known  -  it  matters  to  patent  attorneys  but  little  to  us,  because  the 
fact  remains,  the  company  invested,  say,  two  man-years  in  the  R&D. 

The  fact  that  our  world  is  still  divided  into  national  territories,  and  that  intellectual  property  rights 
are  protected  by  national  patent  offices  and  in  national  borders,  mean  that  a  patent  protects  an  idea 
in  one  country  and  one  market.  Regional  coverage  of  patent  protection  must  be  deliberately  decided 
by  a  company.  So  when  one  invention,  one  patent  application,  is  filed  at  home,  it  is  a  sign  that  a 
company  intends  to  market  it  in  the  domestic  market  only.  When  patents  are  filed  in  seven  or  eight 
countries,  it  shows  the  company  intends  to  either  manufacture  or  market  the  product  in  many 
countries. 

Patent  analysis  is  difficult.  We  must  treat  the  data  with  care.  Some  years  ago,  the  OECD 
secretariat  in  Paris  published  a  manual,  a  guideline,  on  what  one  should  observe  in  working  with 
patent  documents.^  All  the  possible  mistakes  one  can  make  in  analyzing  patents  are  listed,  so  that 
one  can  prevent  them  if  one  reads  this  document  carefully. 

2.  Positioning  of  firms  in  global  telecom  markets  -  an  example  of  innovation 
benchmarking 

Let  us  begin  our  study  of  innovation  benchmarking  with  companies  in  the  area  of  telecom 
manufacturing.  The  companies  in  our  analysis  are  listed  in  Table  1.  Our  objective  is  to  examine  the 
marketing  information  inherent  in  patent  statistics. 

Let  us  examine  Siemens  first.  Figure  1  shows  the  number  of  patent  documents  originating  with 
Siemens  (they  have  a  number  of  affiliated  companies),  somewhere  in  the  world,  and  we  first  look 
at  the  domestic  market.  The  number  of  patents  filed  is  the  largest  in  Germany.  Many  of  them 
remain  only  in  the  domestic  market.  But  a  consiuerable  share  of  all  invention^  ^.i^'  -  in  the 
United  Kingdom.  So  out  of  all  countries  in  the  world,  the  U.K.  is  the  most  preferred  foreign 
market  for  Siemens  in  terms  of  protection  of  their  inventions. 


^  OECD  (ed.).  The  Measurement  of  Scientific  cuuJ  Technological  Activities:  Using  Patent  Data  as  Science  cbkI 
Technology  Indicators.  Patent  Manual  1994,  OECD/GD  (94),  114,  Paris  1994.  A  comprehensive  bibliography 
on  patent  analysis  is  included  in  this  source. 


Table  l' 
Analyzed  Telecoms  Manufacturers  and  Network  Operators. 


Company 

No.  of  European 
patent  applications  in 
telecom   technology 
invented    1987-89 

Communications 
equipment  (US  $  m 
resp.  international 
communications 
revenue    1993) 

Alcatel  NV 

423 

14,823 

AT&T 

378 

11,801 

Bosch 

226 

3,530 

Ericsson 

82 

7,767 

Fujitsu 

288 

4,774 

GEC 

181 

1,948 

Hitachi 

83 

1,555 

IBM 

342 

5,299 

Matshushita 

100 

2,227 

Motorola 

403 

10,096 

NEC 

419 

9,480 

Nokia 

78 

2,355 

Northern  Telecom 

110 

7,860 

Oki 

27 

1,590 

Philips 

378 

1,868 

Siemens 

552 

12,205 

Sony 

100 

1,181 

STET/Italtel 

43 

1,520 

Thomson 

132 

n.a. 

Toshiba 

222 

1,710 

Bellcore 

82 

0  (domestic) 

British  Telecom  (BT) 

121 

3,193 

France  Telecom  (F 1 ) 

89 

3,693 

GTE-Sprint 

39 

1,188 

NIT 

86 

0  (domestic) 

*  Data  sources  for  patent  statistics  for  (consolidated)  company  affiliations  are  Schmoch,  U.,  and  Th.  Schnoring, 
Technological  strategies  of  telecommunications  equipment  manufacturers,  Telecommunications  Policy  18  (5), 
pp.  397-414,  1994,  and  lengthier  German  data  annexes  cited  therein  (from  1992).  Communications  equipment 
revenues  are  from  Communications  Week  International,  pp.  16-17,  Nov.  1995. 
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Figure  1 .'  Destination  countries  of  telecommunication-related  patent  applications  by  Siemens 
(consolidated)  1987-1989. 


This  is  explained  by  Siemens'  serious  effort  to  enter  the  British  market,  in  part  through  patenting, 
in  the  late  1980's.  Other  large  European  countries  are  nearly  equally  covered  with  patents.  For 
smaller  countries,  the  patent  applications  declines.  The  number  of  intellectual  property  rights  in  the 
domestic  market  is  less  than  double  those  abroad  -  so  the  company  is  quite  intemational  in  its 
perspective.  In  the  United  States,  though  it  is  the  single  largest  market  in  the  world,  the  number  of 
duplicated  patents  remains  low  despite  the  company  Rolm  which  was  acquired  there  producing 
some  inventions.  Siemens  has  neglected  this  country  in  comparison  with  Europe,  and  neglected 
Japan  as  well. 

We  can  carry  out  a  similar  analysis  for  other  companies  as  well.  Table  2  provides  similar 
information  for  the  other  companies.  In  this  table,  the  number  of  patents  filed  in  the  foreign 
country  with  the  most  patents  applied  for  is  taken  as  the  benchmark  value  of  1.0. 

It  is  evident  from  Table  2  that  GEC  is  the  mirror  image  of  Siemens  -  a  British  firm  filing  heavily  in 
Germany;  GEC  and  Siemens  run  a  joint  subsidiary,  originating  from  the  Plessey  group.  Note  the 
behavior  of  Japanese  companies.  This  is  a  statistical  artifact.  The  patent  law  in  Japan  is  such  that 
they  cannot  easily  accumulate  and  combine  several  claims  to  be  protected  in  one  document.  In 
practise,  each  claim  requires  its  own  document.  Experienced  patent  lawyers  provide  a  rule  of 
thumb  -  divide  the  number  of  Japanese  domestic  patent  applications  by  six  or  so  to  arrive  at 
roughly  comparable  numbers.  So  we  cannot  surmise  that  the  patenting  activity  in  Japan  is  as  fast 
and  furious  as  the  numbers  indicate. 


Source:  Schmoch,  U.,  International  Patentin;;  .Strategies  of  Multinational  Concerns:  The  Example  of 
Telecommunications  Manufacturers,  in  OECD  (ed).  Innovations.  Patents  and  Technological  Strategies,  pp.  223- 
237.  Paris.  1996. 
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To  this  point,  the  strategic  marketing  aspect  of  innovation,  as  revealed  in  patent  apphcations  by 
protected  national  markets,  has  been  analyzed.  It  v/as  seen  that  potential  strategic  initiatives  of 
companies  in  foreign  markets  can  be  tracked.  In  analyzing  these  data,  one  arrives  at  the 
conclusion,  that  in  the  telecom  industry  there  are  several,  very  different  strategic  positionings  of 
companies  at  the  end  of  the  1980's  (see  Table  3).  There  is  a  group  of  companies  from  various 
countries,  which  have  an  average  share  of  broad  patents.  We  have  another  group  of  companies, 
selective  in  patenting  their  inventions  abroad  among  them  some  Japanese  manufacturers.  Then  we 
may  discern  a  group  of  companies  with  a  special  focus  on  the  American  market.  Finally,  there  are 
companies  with  special  focus  on  the  Japanese  market.  And,  another  group  of  companies  -  among 
them  newcomers  in  that  market  -  that  do  little  on  the  Japanese  market.  What  do  we  know  about 
how  these  companies  position  themselves  in  the  overall  information  technology  arena,  with  their 
telecom  activities?  Are  they  broad,  or  narrow,  in  technological  terms?  This  is  a  typical 
multidimensional  problem.  Here  we  offer  a  brief  introduction  to  a  statistical  technique  known  as 
multidimensional  scaling.  Suppose  you  were  given  a  typical  triangle  of  road  distances  between 
pairs  of  major  European  cities.  For,  say,  8  cities,  there  are  8  x  7  /  2  =  28  such  distances.  Now, 
suppose  you  were  asked  to  place  these  cities  on  a  two  dimensional  map,  such  that  the  distance 
between  each  pair  of  cities  precisely  matches  the  distance  noted  in  the  table.  The  task:  Write  a 
computer  algorithm  that  will  do  so.  There  are  such  algorithms,  and  they  position  each  "city" 
(which  in  some  cases  is  a  variable,  technology,  or  a  company),  state  whether  the  "map"  is  accurate 
or  not  (in  terms  of  a  coefficient  of  goodness  of  fit),  and  provide  other  types  of  useful  information. 
This  is  multidimensional  scaling  (MDS),  a  version  of  which  is  also  known  as  smallest  space 
analysis  (SSA).  Note  that  there  is  an  exact  solution  only  in  the  (28-1)  dimensional  space,  as  our 
starting  point  is  road  distances  not  air  distances  taken  from  the  two-dimensional  surface  of  our 
globe  as  usual.  Let  us  conduct  an  MDS  analysis  of  IT.  We  take  the  telecom  manufacturers 
(including  for  comparison  the  network  operator  in  Japan,  NTT),UO  use  their  patent  profiles  over 
technological  entities  (a  fine  classification  exists,  including  more  than  70,000  individual  items,  the 
International  Patent  Classification),  and  define  six  major  fields  in  information  technology,  telecom 
(TELCOM),  electronic  elements  (ELTRN),  multimedia  technology  (or  audio-visual  or  consumer 
electronics,  (AVEL),  optical  technology  (OPTICS),  storage  (STOR)  and  data  processing  (DAT). 
We  then  compare  the  profiles  of  any  two  companies  to  see  whether  they  are  similar  or  not. 

Table  3 
Typical  patent  strategies  of  selected  companies  on  foreign  markets.^ 


Feature 

Examples 

Average  share  of  foreign  patents,  broad  coverage 

Alcatel,  AT&T,  Philips,  Siemens 

Generally,  little  foreign  patenting,  but  broad 
coverage 

Ericsson,  GEC,  Motorola,  many  network 
•operators 

Selective  strategy 

Hitachi,  Oki,  Matsushita,  Nor  lei 

Special  focus  on  the  American  market 

Matsushita,  Hitachi,  NEC,  Thomson,  Philips 

Special  focus  on  the  Japanese  market 

AT&T,  IBM,  Motorola,  NorTel,  Philips 

Low  presence  on  the  Japanese  market 

Bosch,  Nokia,  Siemens,  STET,  Thomson 

'  This  is  justified,  as  the  NTT  labs  develop  IT  technology  in  collaboration  with  Japanese  manufacturers  which  is 
patent-protected  by  NTT;  see  Grupp,  H..  Efficiency  of  government  intervention  in  technical  change  in 
telecommunications:  Ten  national  economies  compared.  Technovation  13,  (4),  pp.  187-220,  1993. 


Source:    Schmoch.    U.    and    Th.    Schnoring.    Technological    strategies    of    telecommunications    equipment 
manufacturers,  Telecommuiiicauons  Policy  iS(?).  pp.  397-413,  1994. 


(We  calculate  the  correlation  coefficient  of  each  pair  of  company  technology  profiles).  Two 
companies  which  each  put  16.6  %  of  patenting  activity  in  each  of  six  fields  will  have  a  correlation 
of  one.  Two  companies  each  of  which  puts  100  %  of  its  patenting  activity  into  a  different  field, 
will  have  a  low  correlation  (R'  =  0.2  in  the  example).  They  are  thus  considered  dissimilar. 

Figure  2  shows  a  multidimensional  scaling  map  of  companies,  where  distances  in  the  diagram 
represent  similarity  -  Euclidean  proximity.  To  understand  the  MDS  map  fully,  as  with  any  map  we 
need  a  convention  or  wind  rose,  a  "north"  and  "south"  in  IT.  An  artificial  "North"  and  "South"  is 
created  as  follows.  We  invent  an  artificial  company,  one  that  doesn't  exist  and  that  is  active  in  one 
subfield  -  say,  multimedia  or  consumer  electronics.  For  this  field,  we  assign  all  this  imaginary 
company's  patenting  activity,  100  %.  Then  this  company  becomes  a  "pole"  -  one  can  compare  all 
other  companies  to  this  virtual  company  that  is  the  strongest  possible  in  this  field.  We  thus  create 
fictitious  companies  to  represent  the  "pole"  in  optics,  in  electronics  components,  and  so  on.  This 
map  of  technological  profiles,  in  terms  of  several  poles,  represents  real  findings  and  not  artificial 
ones.  Looking  at  single  companies  validates  this  method  -  and  we  have.  This  is  the  simplest  way 
to  benchmark  individual  companies  relative  to  other  firms  -  each  company  can  recognize  their 
closest  competitors  as  those  that  have  the  most  similar  profiles. 
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Figure  2.  MDS  map  of  information  technology  for  selected  companies  in  the  period  1987  to  1989. 


''  Source:  Schmoch,  U..  Evaluation  of  technological  strategies  of  companies  by  means  of  MDS  maps.  Iniemational 
Journal  of  Technology  S4anagemeni  10(4-61.  pp.  426-440.  1995. 


Now,  let  us  zoom  down  to  a  more  limited  market  -  only  telecoms.  We  introduce  an  MDS  map 
based  on  a  breakdown  of  patenting  in  subfields  (Figure  3).  We  get  in  principal  the  same  thing,  but 
just  a  window  of  the  larger  map.  We  see  groupings  of  companies  who  are  strong  in  optical 
telecommunications  (OPT  T),  switching  (SWIT),  mobile  radio  telecoms  (RATIO  T)  and  electrical 
transmission  (ELKTT  -  remote  measuring  and  sensing),  and  terminals  technology  (TRML).  The 
map  provides  us  with  an  interesting  picture:  we  now  know  who  are  the  strong  innovators  in 
optics,  in  transmission  -  Toshiba  is  a  newcomer,  so  we  get  new  information  on  new  entrants  in 
this  field  where  we  may  not  have  subject  information  before.  The  network  operating  companies  - 
with  the  exception  of  NTT  -  support  the  national  innovation  systems  mainly  with  optical 
technology,  an  important  ingredient  to  modem  telecoms  networks.  Whereas  at  the  end  of  the 
1980's,  everyone  was  competing  with  everyone,  at  the  beginning  of  the  decade,  there  were  islands 
of  specialization,  a  nice  partitioning  of  the  markets.  After  the  opening  of  the  telecom  markets 
through  deregulation,  there  was  fierce  competition  in  nearly  ail  markets.  This  is  what  a  similar 
MDS  map  would  show,  for  the  beginning  of  the  1980's.  From  several  snapshots  of  the  MDS 
strategic  positions,    you   can   make   a   movie,   combining    them,   to   get   a   dynamic   picture. 
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Figure  3.  MDS  map  of  communications  technology  for  selected  companies  in  the  period  1987-1989.'° 


Source  as  in  t'ootnoie  ot  Figure  2.  Network  operators  underlined. 
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3.  Knowledge  production  and  growth  -  the  innovation  module 

Timelines  of  technical  results  and  newness  of  the  company's  technology  portfolio  strongly  affects 
its  innovation  performance  and  -  more  specifically  -  new  product  revenues."  The  lessons  for 
clarifying  the  role  global  technology  knowledge  plays  for  the  technology  levels  to  be  achieved  are 
two-fold.  First  of  all,  in  general  terms,  the  more  patents  with  international  significance  a  company 
takes  out  the  more  sophisticated  its  product  innovations  seem  to  be.  On  the  other  hand,  some 
companies  offer  very  sophisticated  products  on  world  markets  with  no  comparable  patent 
production.'^ 

There  are  three  possible  explanations  for  the  latter  observation.  First,  companies  may  have  a  very 
good  tacit  in-house  knowledge  base  in  the  relevant  technology,  or  rely  on  secrecy  or  very  short 
market  introduction  times  and  do  not  care  for  comprehensive,  international  protection.  Another 
possibility  would  be  that  companies  have  a  strong  domestic  patent  base  but  do  not  take  out  foreign 
duplications  of  their  inventions,  accepting  all  the  associated  international  market  risks.  This  case 
can  be  checked  by  an  analysis  of  patent  flows  (Table  2)  and  can  be  ruled  out.  The  third  possibility 
is  that  the  companies  produce  excellent  products  from  global  knowledge  external  to  the  company. 
By  acquiring  leading-  edge  companies,  licensing,  networking  and  other  forms  of  technology 
cooperation  they  may  produce  innovative  products  from  creative  technologies  of  other  firms 
(including  public  laboratories).  This  is  an  important  element  in  telecommunications.'^  From  this 
analysis  it  is  concluded  that  there  are  various  ways  to  innovation.  Some  companies  acquire 
technological  knowledge  from  other,  e.  g.  global  sources  instead  of  using  intramural  technology 
generation  and  patent  protection.  But  not  all  companies  can  do  so  in  telecoms,  so  that  for  a  number 
of  companies  a  knowledge  production  relationship  between  in-house  technology  generation  and 
innovations  achieved  should  appear  to  be  established.  Furthermore,  technology  generation 
anticipates  innovation  performance  levels  for  some  years.  Patent  stock  data  for  previous  years 
should  fit  better  to  the  technology  levels  than  the  most  recent  activities  as  cumulative  technology 
acquisition  by  firms  is  so  important.  The  "knowledge  production  function"  for  innovation  as 
measured  by  the  growth  levels  of  innovative  products  can  be  modeled  as  follows  (this  is  a  further 
development  of  the  knowledge  production  function  model  of  Griliches):''' 

It  is  a  one  dimensional  approach  taking  some  scalar  output  measures  as  is  shown  below.  If  we, 
however,  want  to  use  qualitative,  non-pecuniary  proxy  measures,  i.  e.  for  product  quality,  we 
cannot  use  the  conventional  production  models.  A  version  of  linear  programming  exists,  however, 
that  was  explicitly  built  to  measure  the  efficiency  of  decision-making  units  (which  can  be 
individual  firms)  and  that  does  allow  qualitative  inputs.  This  approach  is  known  as  Data 
Envelopment  Analysis  (DEA).  Essentially,  it  examines  which  decision-  making  units  (DMUs)  are 
on  their  production  possibilities  frontier,  or  isoquant  in  the  knowledge  economy  and  which  are 
not.  "  Here  we  try  out  how  far  the  scalar  approach  holds. 


'   Roberts,  E.B.  Benchmarking  the  Strategic  Management  of  Technology:  II  -  R&D  Performance.  MIT  Sloan 
School  of  Management.  International  Center  for  Research  on  Management  of  Technology,  WP  #1 19-95,  1995. 

■  See  H.  Grupp.  Technical  change  in    a  global    market:   competition   in    solar  cell   development.   23rd  annual 
E.A.R.I.E.  Conference,  Vienna,  September  1996. 


ij 


Compare  the  national  R&D  infrastructures  in  Grupp  ( 1993,  loc.  cit). 


'■*  Summarized  in  Griliches.  Z.,  R&D  and  Productivity:  Econometric  results  and  measurement  issues,  in  Stoneman, 
P.,  ed..  Handbook  of  the  economics  of  innovation  and  technological  change,  Basil  Blackwell,  Oxford,  pp.  52-89. 
1995. 

"  See.  e.g.,  H.  Grupp  (1996,  ioc.  cit.),  or  Grupp..  H..  S.  Maital,  A.   Frenkel  and  K.   Koschatzky.  The  relation 
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The  knowledge  production  function  approach  can  be  represented  in  the  following  way: 

Y  =  a(t)KP+u 

where  Y  is  some  measure  of  output  of  the  firm,  K  is  a  measure  of  cumulated  knowledge  or 
research  "capital",  a(t)  represents  other  determinants  which  affect  output  and  vary  over  time 
including  standard  economic  inputs  such  as  capital  investment,  labor  and  so  forth  while  u  reflects 
all  other  random  fluctuations  in  output. 
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Certainly,  this  is  just  a  first  approximation  to  a  considerably  more  complex  relationship. 

From  the  logarithmic  form  we  arrive  at  the  growth  equation 

d  log  Y  /  dt  =  (irf)  dy  /  dt  =  a  +  p  (RA')  +  du/dt 

where  the  term  (d  log  K)/dt  is  replaced  by  using  the  definitions  p  =  dY/dK  =  P  (Y/K)  and 
R=dK/dt  for  the  net  investment  in  knowledge  capital.  We  now  calculate  the  deflated  growth  in 
communications  equipment  revenues  1993  in  comparison  to  1986  '^  and  approximate  R  by  the 
number  of  patent  applications  following  the  base  year  1986,  i.e.  inventions  in  the  priority  years 
1987-89.  This  is  a,  "skeletal"  model  of  depreciation  and  obsolescence  of  (patented)  knowledge, 
but  more  realistic  data  are  difficult  to  obtain.  It  means,  that  inventions  from  1986  or  earlier  years 
do  no  more  matter  for  the  revenues  in  1993,  and  inventions  from  1990  and  later  do  not  yet.  Patent 
application  number  always  measure  the  increase  (dK/dt)  in  knowledge  as  they  add  up  to  the 
already  existing  (and  eventually  patented)  knowledge.  The  assumed  lag  of  about  four  years  until 
novel  knowledge  affects  markets  is  taken  from  earlier  empirical  investigafions.  '^  Cross-section 
linear  regression  analysis  of  the  19  manufacturing  companies  in  Table  1  gives  the  results  as 
displayed  in  Table  4. 

Knowledge  production  explains  parts  of  the  variance  significantly  although  all  the  other  potential 
inputs  (labor,  physical  capital,  tacit  knowledge)  are  included  in  the  residuals  only.  A  visual 
impression  of  the  relation  is  provided  in  Figure  4. 


between  technological  excellence  and  export  sales:  A  data  envelopment  model  and  comparison  of  Israel  to  EC 
countries,  Research  Evaluation  2  (2),  pp.  87-101,  1992. 

"■  Griliches,  loc.  cit.,  p.  55. 

"  Using  OECD's  implicit  GDP  price  indexes. 

'"  Grupp.  H.,  Innovation  Dynamics  in  OECD  Countries:  Towards  a  Correlated  Network  of  R&D  Intensity,  Trade. 
Patent  and  Technomeiric  Indicators,  in  OECD  (ed.).  Technology  and  Productivity  -  the  Challenge  for  Economic 
Policy.  Paris,  pp.  275-295,  1991. 
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Table  4 

Regression  results  for  the  knowledge  production  function  of  telecom 

manufacturers  (revenue  growth  1993  compared  to  1986). 


Measure  /  Variable 

Value 

P 

0.0053  ±0.0018 

Constant 

0.6312  ±0.3700 

R-  adjusted 
F 

0.298 
8.652 

t 

2.941 

Significance 
DW 

0.91  % 
2.296 
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Figure  4.  Deflated  growth  of  corporate  revenues  and  knowledge  production  of  telecom 
manufacturers. 
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strong  R&D  performers.  They  failed  to  convert  this  into  innovations  that  led  to  average  growth  of 
revenues  in  this  particular  market  for  telecoms  goods.  This  is  not  to  talk  managers  into  a  reduction 
in  R&D  activities,  rather,  to  adjust  the  innovation  "module"  in  a  more  effective  way  to  reach  better 
yields  of  knowledge  investments.  Newcomers  in  the  telecoms  market  such  as  Matsushita  or  Nokia 
grew  so  quickly  with  modest  own  knowledge  sources"  that  one  is  tempted  to  express  a  word  of 
warning:  Long-term  sustainable  growth  may  be  vulnerable  if  you  depend  too  much  on  external  or 
tacit  knowledge  sources. 

4.  Technometric  Benchmarking  for  Individual  Products:  The  Case  of  Optical 
Communications   Lasers 

After  examining  broader  industry  trends,  it  is  possible  to  "zoom  down"  to  the  product  level,  using 
a  different  approach.  Here,  patent  statistics  cannot  help.  In  an  invention,  it  is  not  specified  what  the 
product  characteristics  will  be  like.  We  need  another  instrument  of  analysis  of  product  innovation 
quality  -  one  known  as  technometrics.  We  have  the  following  starting  point.  A  product  is 
described  by  its  characteristics.  Consumers  do  not  buy  "products",  rather  they  purchase  a 
combination  of  characteristics  or  attributes,  that  satisfy  their  wants  and  needs.  For  example,  a  laser 
is  described  by  wave  length,  power,  stability,  and  so  on.  In  general,  experts  know  which  are  the 
important  characteristics.  But  the  problem  is,  each  characteristic  has  a  different  unit  of 
measurement.  So  it  is  not  a  vector  -  you  cannot  derive  a  comparable  measure. 

The  technometric  concept  converts  it  into  a  metric  scale,  a  dimensionless  one,  all  features  between 
zero  and  one  -  which  enables  us  to  build  a  profile  that  can  be  compared  one  to  another,  one 
attribute  to  another,  one  product  to  another."" 

Figure  5  provides  a  graphical  illustration  of  technometric  profiles  for  seven  different  products,  and 
three  different  possible  patterns.  Technometric  benchmarking  makes  it  possible  to  construct  a 
detailed  profile  of  the  product,  comparing  one  characteristic  only  across  products.  One  can  also 
look  at  the  entire  profile  of  a  single  product,  across  all  characteristics,  without  weights  -  simply 
draw  all  the  0, 1  values.  Only  if  you  want  a  one-dimensional  scalar  number,  to  aggregate  the 
technometric  scores,  does  one  need  weights  for  each  prodattribute.  The  weights  are,  of  course, 
representative  of  the  preferences  of  customers.  They  can  be  determined  by  market  surveys,  focus 
groups,  or,  at  times,  by  eliciting  the  opinions  of  those  engaged  in  direct  marketing  of  the  product. 


'''  The  growing  importance  of  acquiring  technology  from  outside  sources  is  undeniable;  see,  e.g.,  the  benchmarking 
study  by  Roberts,  E.B.,  Benchmarking  the  Strategic  Management  of  Technology  -  I.  MIT  Sloan  School  of 
Management,  ICRMOT,  Working  Paper  #  1 15-94,  Cambridge,  MA..  1994. 

""  Earlier  concepts  of  technometrics,  such  as  H.  Grupp..  The  measurement  of  technical  performance  of  innovations 
by  technometrics  and  its  impact  on  established  technology  indicators.  Research  Policy  23,  pp.  175-193,  1994,  or 
Frenkel.  A.,  Th.  Reiss.,  S.  Maital,  K.  Koschatzky,  and  H.  Grupp,  Technometric  Evaluation  and  Technology 
Policy;  The  case  of  Biodiagnostic  Kits  in  Israel.  Research  Policy,  23,  pp.  281-292,  1994,  were  not  focused  on 
benchmarking.  The  technometric  procedure  applied  to  benchmarking  is  best  described  in  Shoham,  A.,  H.  Grupp, 
S.  Lifshitz,  and  S.  Maital,  'Technometric  benchmarking:  Identifying  sources  of  superior  customer  value." 
forthcoming,  1996.  We  omit  the  mathematical  expressions  here;  see  the  above  sources  for  these. 
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Figure  5.  Sketch  of  technometric  benchmarking  concept. 
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Figure  6  provides  a  profile  of  product  quality  for  laser  diodes,  of  mm  range,  for  telecom 
applications  in  optical  fibres,  presented  in  the  following  way:  The  world  state-  of-the-art  level  is  set 
to  one.  This  changes  over  the  course  of  time,  but  is  equal  to  one  at  a  given  point  in  time.  For 
Japan,  all  Japanese  manufacturers  are  aggregated,  as  if  they  were  a  single  firm  ("Japan 
Incorporated").  In  some  attributes,  they  are  world-  class,  in  others,  well  below  it.  In  laser  power, 
at  least  one  Japanese  manufacturer  offers  world-class  quality.  In  others,  no  Japanese  manufacturer 
attains  world-class  sophistication.  This  holds  for  all  Japanese  companies  taken  together. 
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Figure  6.  Example  for  product  quality  measurement  for  single  products  and  domestic  market 
aggregation:  Laser  diode  in  the  |im  range  for  optical  communications. 


The  broken  line  portrays  a  specific  company,  Company  A.  It  has  products,  laser  diodes,  on  a 
world  level,  in  part,  and  below  world  level,  in  part.  One  can  see  precisely  strengths  and 
weaknesses  at  one  glance.  So  this,  of  course,  is  the  adequate  "zoom"  for  benchmarking  Company 
A  against  Japanese  competitors.  This  is  a  typical  application  of  technometrics  fpr  benchmarking  the 
products  of  company  A  for  the  domestic  Japanese  competition.  Of  course,  in  such  a  case. 
Company  A  can  search  for  a  remedy  for  this  situation,  if  it  indicates  weaknesses  -  by  looking 
abroad,  for  those  who  have  a  technological  solution.  In  other  cases,  one  can  search  for  a  strategic 
partner  in  Japan. 
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Figure  7  shows  a  technometric  comparison  of  laser  diodes  in  the  micrometer  range,  not  suitable  for 
optical  communication.  There  is  Company  A,  German,  and  a  line  for  Germany  without  Company 
A,  and  the  world  class  -  the  present  profile  of  all  Japanese  and  all  US  companies.  Company  A  will 
learn  from  this,  whether  there  are  German  competitors  better  than  itself,  and  whether  there  is 
expectation  that  one  can  find  a  partner  for  a  strategic  alliance  in  Germany  -  and  if  not,  to  which 
other  country  might  one  look.  This  is  exceedingly  useful  for  strategic  innovation.  For  an  overall 
measure  of  product  quality  index,  one  dimensional,  we  can  do  this  only  if  we  have  preferences  of 
customers,  showing  what  weights  they  give  each  single  characteristics.  There  are  several  methods. 
You  can  ask  people,  a  sample  of  them,  how  they  would  value  single  characteristics.  You  can 
devise  this  from  prices  -  the  method  of  hedonic  price  indexes  -  by  seeing  the  statistical  link 
between  product  prices  and  their  attributes,  with  coefficients  of  attributes  indicating  the  importance 
of  those  attributes.  Such  information  is  presently  not  available  for  laser  diodes  in  optical 
communications. 
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Figure  7.  Example' of  product  quality  measurement  for  single  company  products  and  international 
comparison:  Laser  diodes  in  the  |im  range. 
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5.  A  Typology  of  Strategic  Focus 

One  can  construct  a  kind  of  typology  of  companies  and  their  technological  strengths  or  weaknesses, 
building  on  the  technometric  profiles.  There  are  four  basic  types  of  firms  (see  Figure  8): 

*  uncompetitive, 

*  unfocused, 

*  focused,  and 

*  dominant. 
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Figure  8.  Typology  of  Firms.  Focused  firms  have  product  strength  for  attributes  consumers  regard 
as  important. 


One  sees  this  by  mapping  firms'  products  in  two-  dimensional  space,  with  the  X  axis  indicating, 
for  each  attribute  of  a  product,  its  technometric  score  (from  zero  to  one),  and  the  Y  axis  indicating, 
for  each  attribute,  the  weight  or  importance  of  that  product  in  the  eyes  of  its  consumers.  In  other 
words,  a  product  with  10  key  attributes  will  be  characterized  by  10  pairs  of  numbers.  The  first 
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number  in  each  pair,  the  X  value,  represents  the  attribute's  objective,  technometric  score,  and  the 
second  number,  the  Y  value,  indicates  the  subjective  consumer  preference  weight. 

The  way  those  10  points  cluster  establishes  a  firm's  competitive  position.  If  the  firm's  product  is 
consistently  weak  relative  to  its  competitors,  it  is  "uncompetitive".  This  is  shown  by  an  essentially 
vertical  line  rising  from  "LO"  product  quality.  Its  market  success  is  highly  dubious.  If  the  firm's 
products  are  technometrically  strong  for  attributes  consumers  rate  as  unimportant,  and  vice-versa  - 
technometrically  weak  for  attributes  consumers  rate  as  important,  then  the  firm's  product  is 
unfocused,  or  rather  misfocused.  Its  market  share  is  unlikely  to  be  high  or  growing.  If  the  firm's 
products  are  strong  for  attributes  consumers  rate  highly,  but  weak  for  attributes  consumers  think 
unimportant  -  the  product  is  focused,  and  market  share  will  be  strong. 

Finally,  if  the  product  is  technometrically  superior  for  all  its  attributes  -  then  the  product  is  defined 
as  dominant".  The  first  type  of  firm,  "uncompetitive",  has  uniformly  low  product  quality,  for 
attributes  consumer  value  highly  as  well  as  for  those  they  value  less  highly.  These  firms  are 
uncompetitive,  unless  their  products  compete  on  the  basis  of  very  low  price  (and  hence,  are 
produced  at  low  cost).  The  second  type  of  firm  is  "focused".  These  firms  are  strong  precisely  in 
attributes  that  the  market  values  highly.  Their  R&D  tends  to  be  well  directed  and  strategically 
planned  in  line  with  market  preferences.  The  third  type  of  firm  is  "unfocused"  -  their  product 
quality  is  strong  precisely  for  attributes  the  market  as  relatively  unimportant,  perhaps  as  a  result  of 
poor  R&D  investment. 

Finally,  there  are  dominant  firms.  These  firms  have  consistently  high  product  quality  across  all 
attributes,  both  relatively  important  and  relatively  unimportant  ones.  They  tend  to  dominate  their 
markets.  We  anticipate  a  positive  link  between  the  performance  of  companies  and  their  products, 
and  their  placement  in  the  above  typology.  Uncompetitive  or  unfocused  products  should  fare  more 
poorly  than  those  that  are  focused  and  dominant.  As  more  elaborate  data  are  missing  for 
telecommunications,  we  presently  cannot  provide  examples.  Such  analysis  remains  on  the  research 
agenda;  the  feasibility  of  this  strategic  analysis  has  already  been  shown  for  sensor  technology.  ^' 

Conclusion 

This  is  only  a  preliminary  study  of  a  new  approach  to  innovation  benchmaiking,  as  applied  to 
telecom  and  information  technology.  Management  always  should  begin  with  measurement.  This  is 
especially  true  of  the  difficult  and  risky  task  of  managing  innovation.  By  quantifying  aspects  of  the 
innovation  process,  hopefully  management  decisions  can  become  fact-based  and  hence  lead  to 
superior  performance. 


Grupp,  H.  and  S.    Maital.  Interpreting  the  sources  of  market  value  in  a  capital  goods  market  -  The  case  of 
industrial  sensors.  R&D  Management,  forthcoming  1996. 
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